Papers by Xin Eric Wang
What’s Missing in Vision-Language Models? Probing Their Struggles with Causal Order Reasoning (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing benchmarks often include a mixture of reasoning questions, making it difficult to truly assess VLMs’ causal reasoning abilities. |
| Approach: | They propose two new benchmarks specifically designed to isolate and rigorously evaluate VLMs’ causal reasoning abilities. |
| Outcome: | The proposed benchmarks show that vision-language models perform poorly on causal reasoning tasks, often only marginally surpassing random guessing. |
Worse than Random? An Embarrassingly Simple Probing Evaluation of Large Multimodal Models in Medical VQA (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Multimodal Models (LMMs) have demonstrated impressive performance on existing medical visual question answering benchmarks. |
| Approach: | They evaluate large multimodal models that perform worse than random guessing on medical questions . authors suggest more robust evaluation methods to ensure reliability of LMMs . |
| Outcome: | a new study shows that large multimodal models perform worse than random guessing on medical visual question answering benchmarks. |
LLM-Coordination: Evaluating and Analyzing Multi-agent Coordination Abilities in Large Language Models (2025.findings-naacl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated emergent common-sense reasoning and Theory of Mind (ToM) capabilities, making them promising candidates for developing coordination agents. |
| Approach: | They propose to use Large Language Models (LLMs) to analyze coordination models in Pure Coordination settings where agents must cooperate to maximize gains. |
| Outcome: | The proposed benchmark evaluates LLMs through two distinct tasks: Agentic Coordination and Coordination Question Answering. |
GUI-Bee: Align GUI Action Grounding to Novel Environments via Autonomous Exploration (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent work of GUI action grounding fine-tunes data from pre-trained MLLMs, but data is limited to specific GUI environments. |
| Approach: | They propose to use a GUI-based agent to collect environment-specific data and fine-tune GUI grounding models with the collected data. |
| Outcome: | The proposed model can be extended to other GUI environments to improve performance. |
Dynamic Evaluation for Oversensitivity in LLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing benchmarks rely on static datasets that degrade over time as models evolve, leading to data contamination and diminished evaluative power. |
| Approach: | They construct a framework that generates model-specific challenging datasets and aggregates them across diverse LLM families. |
| Outcome: | The framework captures emerging defensive patterns and aligns with each model’s unique behavior. |
Hidden in Plain Sight: Reasoning in Underspecified and Misspecified Scenarios for Multimodal LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Multimodal large language models are increasingly deployed in open-ended, real-world environments where inputs are messy, underspecified, and not always trustworthy. |
| Approach: | They evaluate multimodal large language models in real-world environments where inputs are messy, underspecified, and not always trustworthy. |
| Outcome: | The proposed models fail to detect hidden issues even when they possess the necessary perceptual and reasoning skills. |
Multimodal Inconsistency Reasoning (MMIR): A New Benchmark for Multimodal Reasoning Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing Multimodal Large Language Models (MLLMs) are predominantly trained on consistent visual-textual inputs, leaving open the question of whether they can handle semantic mismatches in layout-rich content. |
| Approach: | They propose to use multimodal inconsistency reasoning to assess MLLMs' ability to reason about semantic mismatches in webpages, presentation slides, and posters. |
| Outcome: | The proposed model outperforms open-source models in detecting inconsistencies in webpages, presentation slides, and posters while remaining vulnerable to inconsistent errors. |
SafeKey: Amplifying Aha-Moment Insights for Safety Reasoning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Reasoning Models (LRMs) introduce a new paradigm of explicitly reasoning before answering, but they pose great safety risks against harmful queries and adversarial attacks. |
| Approach: | They propose a safety aha moment that activates safety reasoning and leads to a safe response. |
| Outcome: | The proposed model can generalize to unseen jailbreak prompts while maintaining general abilities. |